課程大綱

課程資訊

課程名稱	分散式機器學習系統 Distributed Machine-Learning System
開課學期	108-2
授課對象	電機資訊學院資訊工程學研究所
授課教師	周承復
課號	CSIE5319
課程識別碼	922 U4430
班次
學分	3.0
全/半年	半年
必/選修	選修
上課時間	星期二7,8,9(14:20~17:20)
上課地點	資111
備註	總人數上限：30人
Ceiba 課程網頁	http://ceiba.ntu.edu.tw/1082CSIE5319_DMLS
課程簡介影片
核心能力關聯	核心能力與課程規劃關聯圖
課程大綱
為確保您我的權利,請尊重智慧財產權及不得非法影印
課程概述	Distributed Machine-Learning System is an introduction to these system-focused aspects of machine learning, covering guiding principles and commonly used techniques for scaling up to large data sets. That is, we will cover the techniques that lie between a standard machine learning course and an efficient systems implementation. Topics will include stochastic gradient descent, acceleration, variance reduction, methods for choosing metaparameters, parallelization within a chip and across a cluster, popular ML frameworks, and innovations in hardware architectures.
課程目標	We look at the performance as well as design issues of large-scale machine learning application that is deployed in practice. After taking this course, students, who basic models and the basic algorithms, are able to modify those models (or systems) in a bunch of different ways such that the systems could run faster and more efficiently. That is, these modifications are really important—they often are what make the system tractable to run on the data it needs to process.
課程要求	待補
預期每週課後學習時數
Office Hours
指定閱讀	待補
參考書目	待補
評量方式 (僅供參考)

課程進度

週次	日期	單元主題
第1週	03/03	Course introduction
第2週	03/10	DNN Overview
第3週	03/17	Techniques for SGD
第4週	03/24	Techniques for SGD 課程影片 https://drive.google.com/open?id=1wwRTT_C4DdbSTZlhRfGnzRkE-7rfodHx
第5週	03/31	Variance Reduction in SGD
第6週	04/07	Markov Decision Process and Reinforcement Learning
第8週	04/21	期中考
第9週	04/28	1. 期中考檢討 2. RL 上半部
第10週	05/05	RL下半部
第11週	05/12	Paper presentation (1) Adam: A Method for Stochastic Optimization (2) Random Search for Hyper-Parameter Optimization (3) Lookahead Optimizer: k steps forward, 1 step back (4) Random Features for Large-Scale Kernel Machines
第12週	05/19	Paper presentation (1) Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (2) A Survey on Transfer Learning (3) Domain-Adversarial Training of Neural Networks (4) Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
第13週	05/26	Paper presentation: (1) The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (2) Human-level control through deep reinforcement learning (3) Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization (4) Mastering the game of Go with deep neural networks and tree search
第14週	06/02	Paper presentation: (1) Applications of Deep Learning and Reinforcement Learning to Biological Data (2) Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks (3) Learning Transferable Architectures for Scalable Image Recognition
第15週	06/09	Paper presentation: (1) Federated Learning for Ultra-Reliable Low-Latency V2V Communications (2) Federated Learning via Over-the-Air Computation (3) Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks (4) HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
第16週	06/16	Paper presentation and term project presentation (1) When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning (2) In-Datacenter Performance Analysis of a Tensor Processing Unit